A SENTIMENT ANALYSIS-BASED SMARTPHONE APPLICATION TO CONTINUOUSLY ASSESS STUDENTS’ FEEDBACK AND MONITOR THE QUALITY OF COURSES AND THE LEARNING EXPERIENCE IN EDUCATIONAL INSTITUTIONS

 

Sarah A. Alkhodair

 Assist. Prof. Dr., Information Technology Department, CCIS, King Saud University, Saudi Arabia, salkhudair@ksu.edu.sa

 

Abstract

The quality of education in a specific educational institution is directly reflected in the outcomes of their system. Higher-quality educational systems continue to deliver better learning experiences to enrolled students and better-developed skills and knowledge. To provide high-quality education, an institution must continually monitor its plans, update its courses’ topics and curriculum, and improve teaching facilities and different learning experiences. Students’ opinions and feedback regarding different aspects of a course and their personal learning experience, if properly gathered and analyzed, can be strong indicators of the quality of that course and help identify the areas of satisfaction and dissatisfaction with that course. Highlighting the strengths and weaknesses of each course helps faculty members put, execute, and evaluate a course quality improvement plan in the following semester. Such valuable students’ feedback and opinions about courses are scattered throughout different social media platforms and managed by different discussion groups, usually students. Thus, gathering honest and freely written comments and opinions in one place is challenging. Furthermore, extracting and analyzing courses’ quality and learning experience-related posts is not a trivial task. This study describes the process of designing and developing a smartphone application utilizing Sentiment Analysis techniques to address the problem of gathering, analyzing, and understanding students’ feedback and comments regarding different aspects of courses quality provided by an educational institution. The project’s primary goal is to benefit from student feedback regarding the institution’s courses to continuously assess and monitor the quality of the courses and the students’ learning experiences. A sample representative dataset of students’ unstructured free-text comments and answers to open-ended questions about five different courses over four consecutive semesters was collected, cleaned, and used to develop and test two sentiment analysis models: Naive Bayes in WEKA and a sentiment lexicon-based model named VADER. To further analyze and assess different aspects of the learning experience and courses along with its overall quality, answers to closed-end questions were also analyzed using the 5-point Likert scale. Preliminary results obtained from evaluating the sentiment analysis models show that the Naïve Bayes model achieved 68.7%, 68.8%, 68.8%, and 68.8%, while the VADER model achieved 72.12%, 72.82%, 72.12%, 71.87%, in terms of accuracy, precision, recall, and F1-score, respectively. Performance testing results of the application show that the maximum usage for the CPU is 44%, for the memory is 119 MB, for sending a request on the network 14.7 KB/s, for receiving a response is 226.5 KB/s, and the maximum energy usage is medium. For stress testing, obtained results show that the application can successfully deal with a maximum of 500 random, fast, and abnormal events. For user acceptance testing, users were surveyed to measure their level of satisfaction with the application using the system usability scale. The results show that 100% of users either agreed or strongly agreed that they would like to use the application and be more engaged in assessing the quality of courses. They also indicated that the application is easy to use, quick, and easy to learn. This paper also highlights various challenges and limitations developers face, along with important recommendations for further improvements and future work directions.

Keywords: Quality of Education, Sentiment Analysis, Course Quality Assessment, Student Feedback.